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HERE is considerable current interest 
pe the techniques of programed learn- 
ing, as well as in the experimental investi- 
gation of the nature of behavioral processes 
involved in their use (Lumsdaine & Glaser, 
1960). It has been proposed (Gagné, in 
press) that the learning that transpires in 
the course of administration of learning 
programs may be called productive learning, 
and distinguished in operational terms from 
the reproductive learning of the typical 
verbal learning experiment. Whereas in the 
latter type, performance is usually measured 
on the specific task that is practiced during 
learning, in the former, measures of terminal 
performance are designed to demonstrate 
mastery of the defined class of tasks. For 
example, learning programs are intended ‘to 
establish proficiency in such classes of tasks 
as “adding binary numbers,” or “solving 
linear equations,” and the specific tasks em- 
ployed to measure such performance are 
considered merely representative of the total 


class. 

Individual differences, naturally enough, 
are prominently observable in productive 
learning situations, perhaps even more than 


1 This study results in part from a collaborative 
effort between a project on mathematical concept 
learning, directed by the senior author at Princeton 
University, and the University of Maryland Mathe- 
matics Project, with which both authors have 
been associated. Gratitude is expressed to John 
Mayor and Helen Garstens, the Director and As- 
sociate Director of the latter project, for their 
encouragement and assistance in carrying out this 
study, and also for their forbearance in counte- 
nancing the use of mathematical ideas for which 
they would hasten to disclaim responsibility. In 
part also, this study owes its support to funds 
granted by the Carnegie Corporation of New 
York. The opinions expressed are those of the 
authors, and do not necessarily reflect the views 
of any of these supporting organizations. 
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they are in verbal learning studies of the 
reproductive sort. It is not uncommon, for 
example, to find that fast learners complete 
a learning program in half the time of slow 
learners.*, This fact has sometimes been 
cited (Skinner, 1958) as an advantage to the 
use of programed learning, contrasted to the 
usual classroom learning, in that individuals 
are allowed to proceed at their own pace 
depending upon their initial ability. How- 
ever, there appears to be little if any current 
evidence about the nature of these individ- 
ual differences in completion of learning 
programs, beyond the fact that they occur. 

One hypothesis about the origin of differ- 
ences in rate of conipletion of a learning 
program is that rate of acquisition of the 
successive items (frames) of the program 
is basically determined by an ability which 
may be called “general intelligence.” Testing 
this hypothesis would presumably involve 
the partialing out (experimentally or other- 
wise) of other factors which might affect 
speed of program completion, such as “read- 
ing speed” and “speed set.” This would by 
no means be easy to do. Besides this such 
a hypothesis about general intelligence tends 
to restore an older, original meaning to the 
latter phrase, that is, “learning rate” ability. 
Undoubtedly such a restoration would be 
welcome to many psychologists, if verifica 
tion could be obtained for it. However, it 
would also run contrary to the general trend 
of results obtained over a period of years, 
most of which have failed to obtain evidence 
of a general factor which might be called 


2 This was the finding in an unpublished study 
which utilized the learning program on equation 
solving described herein—Gacne, R. M., & Dick, 
\WV. Learning measures in a self-instructional pro- 


gram in solving equations, 


learning rate ability that is common to 
achievement in a variety of different learn- 
ing situations (cf. Woodrow, 1946). 

An alternative hypothesis to account for 
individual differences in rate of completion 
of, and achievement in, learning programs 
has been proposed (Gagné, in press). This 
is to the effect that such observed differences 
result primarily from the fact that individ- 
uals begin the task of learning with different 
amounts and kinds of knowledge. Knowl- 
edge relevant to any given final task to be 
learned is conceived as a set of subordinate 
capabilities called learning sets (cf. Harlow, 
1949). These are considered to be arranged 
in a hierarchy such that any learning set 
may have one or more learning sets sub- 
ordinate to it in the sense that they mediate 
positive transfer to the given learning set. 
These subordinate sets in turn have other 
learning sets subordinate to them, and so on. 
Each learning set in the hierarchy is rep- 
resented by a distinct class of tasks, and 
measured in the individual by one or more 
representative tasks from this class. In 
order for learning to occur at any point in 
the hierarchy, according to this theory, each 
of the learning sets subordinate to a given 
task must be highly recallable, and inte- 
grated by a thinking process into the solu- 
tion of the problem posed by the task. The 
attainment of the final task is thus conceived 
to be a matter of successive attainment and 
“integration” of a series of lower level 
learning sets, beginning with those which are 
already available to the individual. Failure 
to achieve the final task may have several 
causes: a subordinate learning set may have 
been omitted from the program, and there- 
fore could not have been acquired; insuffi- 
cient practice (or other condition) may have 
resulted in low recallability of one or more 
subordinate learning sets; or the program 
may have been unsuccessful in inducing the 
integration necessary for the attainment of 
any given learning set in the hierarchy. 

The hierarchy of learning sets which sup- 
ports any given final task may be defined 
by an analysis procedure. The question is 
asked of the task, and successively of each 
task thereby defined: “What would the in- 
dividual have to know how to do in order 
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to be able to achieve this (new) task, when 


>)? 


given only instructions?” The answer to 
this question defines one or more subordi- 
nate learning sets, and each of these in turn 
may be seen to be dependent upon one or 
more subordinate learning sets, until the 
entire hierarchy is defined (see Figure 1 
below). At the bottom of this hierarchy are 
some learning sets which are very simple 
and very general indeed. Although they are 
arrived at in the manner described, they 
seem to resemble, as a class, those tasks 
presented by tests used to measure certain 
very general abilities which have been identi- 
fied by factor analysis techniques. For ex- 
ample, a learning set hierarchy for the task 
of adding dissimilar fractions may be found 
to contain at its lowest level such learning 
sets as “adding two-digit numbers,” “multi- 
plying two-digit by one-digit numbers;” and 
“recognizing symbols.” These appear to be 
the tasks represented by tests of Number 
Ability and Symbol Recognition (a variety 
of Associative Memory) (French, 1954). 
We shall examine additional implications of 
this finding in a moment. 

In accordance with this conception of the 
learning set hierarchy, we can begin to ex- 
plore an expected set of correlates (predic- 
tor variables) for the individual differences 
to be observed in connection with the ad- 
ministration of a learning program. First of 
all, individuals would be expected to begin 
a learning program with different numbers 
and kinds of learning sets. Many of these 
will of course be irrelevant to the task, that 
is, they will not have been identified as 
members of the hierarchy derived in the 
manner previously described. More im- 
portantly, however, individuals will differ 
in the pattern of learning sets they possess 
within the hierarchy of relevant learning 
sets. Some may possess only a few, all at 
the lowest level; others may possess nearly 
all that are required to achieve the final per- 
formance. Still others may display rather 
uneven patterns, indicating “gaps” in their 
previously acquired knowledge. The number 
and kind of learning sets that the learner 
brings to the learning program situation 
may be expected to determine how rapidly 
he completes it, other things being equal 
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For an individual who already has many of 
the relevant learning sets, responding to the 
frames of a learning program will be largely 
a matter of review, of traveling over familiar 
ground. For a person with only a few low 
level learning sets to begin with, the attain- 
ment of each new one may be expected to 
take time greater than is needed for “re- 
view”; and more of these will need to be 
acquired in order to attain successful per- 
formance on the final task. 

A second source of individual differences 
has its origin in the fact that different indi- 
viduals approach the learning with different 
patterns of “basic abilities,” which may turn 
out to be those factors which appear con- 
sistently and dependably in factor studies 
(French, 1954). Some of these basic abili- 
ties will be related to the learning sets which 
must be acquired, in the sense that they will 
have been identified as the “bottom row” of 
the learning set hierarchy arrived at by 
means of the analysis just described; others 
will be not so related to the hierarchy. In 
order to have a distinctive set of terms, we 
may call the former “relevant” basic abilities 
and the latter “irrelevant.” In the sense we 
wish to use the term, relevant means related 
by theoretical prediction. 

In summary then, the suggestion derived 
from this theory is that differences in rate of 
completion of a learning program are pri- 
marily dependent upon the number and kind 
of learning sets (i.e., the “knowledge”) the 
learner brings to the situation, secondarily 
upon his standing in respect to certain rele- 
vant basic abilities, and not in any direct 
sense upon a general “learning rate” ability. 


Statement of the Problem 


As is evident from our previous state- 
ments, we recognize the possibilities of in- 
dividual differences at the beginning of a 
learning program to be: (a) differences in 
knowledge, i.e., the number and pattern of 
learning sets, both relevant and irrelevant 
to the final performance; (6) differences 
in amounts of basic abilities, relevant and 
irrelevant; and (c) differences in a general 
learning ability, general intelligence. Looked 
at as a whole, the problem may be stated 
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in the following way: what are the causes 
of individual differences in performance on 
a learning program, and what relative 
weighting of causal effect can be assigned 
to each of them? The implication of the 
theory previously stated is that a substantial 
proportion of the variance in learning pro- 
gram performance is attributable to the at- 
tainment or nonattainment of learning sets 
relevant to the final task which the program 
is designed to teach. 

In more specific terms, the theory to- 
gether with assumptions derived from other 
psychological findings, would predict the 
following things about individual differences 
in programed learning (or, more generally, 
in productive learning) : 

1. Individual differences in those begin- 
ning a learning program may be independ- 
ently measured as differences in (a) general 
intelligence, (b) relevant basic abilities, (c) 
number and pattern of relevant learning 
sets. The word “relevant” means “rationally 
derived as related” in accordance with the 
analysis procedure previously outlined. 

2. An ideally effective learning program 
has the effect of reducing the variance at- 
tributable to lc to zero, since in such a pro- 
gram all learning sets are attained by every- 
one. To the extent that a learning program 
is ineffective, however, an increasing num- 
ber of individuals will be “eliminated” from 
attainment of learning sets at progressively 
higher levels of the hierarchy (and accord- 
ingly, from attainment of the final task). 

3. Both Factors 1) and lc are considered 
to mediate specific, rather than general, posi- 
tive transfer to the learning of relevant 
learning sets in the hierarchy. Positive trans- 
fer in this situation may be measured by 
rate of learning, that is, by the time taken 
by the learner to attain any or all of the 
relevant learning sets including the final task. 
Accordingly, as the learner progresses up- 
wards in the hierarchy, his rate of learning 
should depend increasingly on the attain- 
ment or nonattainment of relevant learning 
sets, and decreasingly on relevant abilities. 
Specifically, this means that the rate of 
learning of learning sets will correlate to a 
decreasing extent with relevant abilities as 
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one progresses to higher and higher learning 
sets. Since the same progression partakes 
increasingly of new learning (as opposed to 
recall of previously acquired learning sets), 
there should also be increasing correlation 
of rate of learning with attainment of im 
mediately subordinate learning sets. 

4. The effects of Factor la, general in- 
telligence, should be apparent in moderately 
low correlations of learning rate with meas- 
ures of this factor. However, the amount of 
“general” transfer will be expected to remain 

nstant as the learner progresses upwards 
in the hierarchy, as exhibited by a constant 

f correlation coefficient with learning 
rate at all levels of the hierarchy. The same 
pattern of correlation (1.e., a constant one) 
would be expected between learning rates 
ind irrelevant basic abilities, presumably be 
cause of the extent to which these 


sample general intelligence and mediate gen- 


too 


eral transfer. 

5. The relations of basic and general abili- 
ties to achievement of learning sets in the 
hierarchy, as opposed to learning rate, ap 
somewhat more difficult to predict. 
By achievement here is meant being success- 
unsuccessful on the task which 
represents a learning set, more or less in- 
lependently of time (within reasonable 
limi Primarily because of the reasoning 

in Paragraph 2, more and more 
individuals would be expected to “drop out”’ 
it higher levels of the hierarchy, to the ex- 
tent that the learning program is ineffective. 
\s this happens, the “selected” group will 
increasing numbers of 
e high on relevant basic 
lesser extent, 


luli or 


presumably contain 


individuals who scot 
| 


abilities (and perhaps to a 


aj 
‘lligence). Consequently, 
vant basic abilities with 
chievement of progressively higher learning 


high on general inte 
correlations of. rele 
sets should exhibit an increasing pattern, in 
a moderately ineffective learning program. 
In a sense, then, the rate and amount of such 
increasing correlation may be looked upon 
as an inverse measure of the effectiveness 
of a learning progran 

It was our intention to test as many as 
possible of these predictions from the theory 


of learning set hierarchies. To do this, we 
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first analyzed a final task represented by an 
existing learning program in solving linear 
algebraic equations, to define a hierarchy of 
learning sets. Before administering the 
learning program to a group of seventh 
grade children, we obtained measures of 
those basic abilities which were revealed by 
this analysis to be relevant, as well as of 
two which were irrelevant. During the ad- 
ministration of the program, records were 
obtained which provided measures of rate 
of learning of each learning set in the hier- 
archy. Following this, measures were made 
of achievement of each learning set, per 
formance of the final task of equation 
solving, and transfer indicated by perform- 
ance in solving equations of unfamiliar form 
and content. Altogether, these measures en- 
abled us to obtain evidence on transfer of 
training within the determined hierarchy of 
learning sets, and on the relation of both 
basic abilities and learning sets (knowledge) 
to rate of learning, achievement, and trans 
fer within the area defined by the learning 


program. 
MrtTHOD 
Analysis of the Task 


The final task 
intended was solving simple linear algebraic 


for which learning was 


equations, either for numerical values of a 
stated variable, or for expressions of a 
stated variable in terms of other variables. 
Examples of these tasks may be found in 
Appendix B. 

Analysis of this task was begun by for- 
mulating an answer to the question: “What 
would an individual have to know how to 
do in order to achieve successful perform- 
ance of this class of task, assuming he were 
given only instructions?” The phrase re 
garding instructions needs some explanation 
(Gagné, in press), which may be restated 
here briefly. The individual must be: told 
the form of the answer (in this case, whether 
numerical or symbolic); informed of any 
definitions of stimuli required; and provided 
with guidance suggesting the application of 


previously acquired learning sets to a new 


1 
task. 
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Assuming this kind of instruction, the 
answer to the question “What would the 
individual have to know?” turned out to be 
three distinct learning sets (tasks). The 
first was “simplifying an equation by adding 
and subtracting terms’; the second was 
“simplifying an equation by multiplying, di- 
viding, adding, or subtracting arithmetic 
numbers”; and the third was “simplifying 
an equation by multiplying and dividing by 
terms.” The concrete meanings of these de- 
scriptions may be grasped by examining the 
tasks used to measure them, listed as I1, I2, 
and I3 in Appendix A; What they repre- 
sent are three types of problem situations 
which the solver of equations meets in the 
course of arriving at a solution to equations 
of the total class. (Of course, there are 
instances in which the simplifications de- 
scribed would actually lead to a solution, but 
these are too simple by themselves to repre- 
sent the total class of linear equation prob- 
lems.) Each of these tasks is distinct: that 
is, it is possible to know how to do any one 
without knowing how to do the others. Each, 
as will become apparent, is based upon dif- 
ferent patterns of subordinate knowledge. 
This analysis was then repeated on each 
of the three learning sets defined as subordi- 
nate to the task (Level 1), until the entire 
hierarchy was defined as shown in ligure 1. 
In this figure, each block contains a descrip- 
tion of a learning set. The large dots under 
each block indicate which of the lines leading 
from subordinate learning sets is “tied into” 
the learning set defined in that block. Some- 
times, a subordinate set several levels down 
the hierarchy is related to a learning set in 
this manner; in such instances, the line de- 
picting the relationship may pass under- 
neath an intervening block, and the absence 
of a dot indicates no “tie-in” with this in- 
termediate learning set. The learning sets 
are arranged into levels indicated by roman 
numerals, primarily for the purpose of de- 
notation. No particular significance is at 
tached to these “levels,” except that the 
learning sets on any given level appear to 
be approximately equivalent in complexity. 
In referring to a particylar learning set, one 
may use the roman numeral to designate 
level and arabic numerals to denote its hori- 
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zontal position beginning with 1 at the left. 
Thus, the three learning sets previously de- 
scribed are I1, I2, and I3. 

In carrying out the analysis, we were of 
course guided by the outline of the learning 
program which was to be used in the study. 
This does not mean, however, that we de- 
rived the learning sets in the hierarchy 
directly from the program—dquite the con- 
trary was the case, and we shall point out 
later that our method of analysis revealed 
two learning sets which were inadequately 
represented in the program. What it does 
mean is that in doing the analysis we ac- 
cepted in general the approach to solution of 
simple equations which had been designed 
into the program. This particular approach 
is by no means the only one. In other 
words, there are perhaps several possible 
learning set hierarchies which could be 
worked out to support this final task, and 
it is quite conceivable that some are “better”’ 
than others in the sense of being more effi- 
cient or more transferable to later learning. 
The proponents of instruction in ‘“‘modern 
mathematics” would almost surely not favor 
this particular one. 

As the theory foresees, the successive ap- 
plication of the analysis procedure defines 
learning sets which are increasingly simple 
and increasingly general in the sense that 
they are potentially supportive of greater 
numbers of superordinate learning sets. A 
learning set like IITA2 (simplifying frac- 
tional expressions), for example, must sup- 
port quite a large number of different tasks 
in the field of arithmetic. The learning sets 
at Level IVA are seen to be almost as simple 
as can be defined, for a human being. The 
concrete form of each learning set may be 
understood by referring to Appendix A. 

Somewhat parenthetically, it needs to be 
stated that the descriptions of learning sets 
employed in Figure 1 employ somewhat old- 
fashioned language, judged against the stand- 
ards of modern algebra texts. The language 
of the learning program was similarly old- 
fashioned. The validity of the procedure, 
however, should not be questioned on this 
account. When the learning sets are under- 
stood as tasks, which can be done by ex- 
amining the problems given in Appendix A, 
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the ambiguities of language are largely re- emphasized that our purpose in this study 
moved, and some intuitive grasp of the was to test a method and some theoretical 
relatedness of the various learning sets in deductions which presumably are applicable 
the hierarchy may be gained. It may also be to any task and any learning program. We 


SOLVING EQUATIONS 


Simplifying an lifying en Simplifying an 
equation by adding uation by mult. equation by multi- 
and subtracting i add.. and Plying and divid- 
terme to both arith. nus- ing both sides by 
sides bers to both sides terre 


Fic. 1. Learning set hierarchy for the task of solving linear algebraic equations. (Large dots at 
the bottom of boxes indicate the relatedness of particular lower level and higher level learning sets, in 
the sense that positive transfer is predicted. At the lowest level, in ovals, are basic ability f 
whose relevance to learning sects in the hierarchy is indicated by arrows.) 
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were not directly interested in the effective- 
ness of the particular “rogram we employed, 
nor in the modernit, /f its content. 

Level 5 of the hierarchy defines tasks 
which we hypothesize to be equivalent to 
those occurring in “factor reference’ tests 
(French, 1954). Number is the name we 
use for tasks requiring the arithmetic opera- 
tions of addition, subtraction, multiplication, 
and division of one- and two-place numbers. 
Symbol Recognition is a variety of Associa- 
tive Memory usually measured by such a test 
as Picture-Number (French, 1954, p. 15). 
Beyond these two familiar factors, we found 
it necessary to identify another which is not 
so well known, called Integrution I (Guil- 
ford & Lacey, 1947; Lucas & French, 1953). 
In its simplest form, it appears to be the 
capability of “holding in mind” several op- 
erations in sequence, and is often measured 
by tests of following directions. It should 
be emphasized that these capabilities (the 
learning sets at Level V) were derived by 
the same analysis procedure as were others 
in the hierarchy. We did not set out to find 
tasks which resemble those used in measur- 
ing basic factors. Instead they represent the 
simplest kinds of things an individual must 
“know how to do” in order to progress up 
through the hierarchy. Having identified 
them, however, it seemed important to gather 
evidence which would relate them as basic 
abilities to performances of higher level 
tasks. 


Materials 


Tests of basic abilities. To obtain measures of 
basic abilities, we used factor reference tests sug- 
gested by French (1954). Number Ability was 
measured by the two tests Addition, and Subtrac- 
tion and Multiplication. Symbol Recognition ability 
(Associative Memory) was measured by means of 
the test Picture-Number. Integration was meas- 
ured by a test called Following Directions.* In 
addition, to measure irrelevant abilities, we used 
two which would provide a rather severe test of 
our hypothesis, Verbal Knowledge (measured with 
a test called Vocabulary, V-1) and Speed of 
Symbol Discrimination (Letter A).4 


8 Supplied by John W. French, Educational 
Testing Service, and used with permission. 

4Used with the permission of the copyright 
owner, Thelma G. Thurstone. 
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Learning program. The learning program de- 
signed to teach the solving of linear equations was 
originally composed of 247 frames. These were 
printed on 4”X 6” cards, and were divided into 
eight convenient and roughly equal sequences, 
which were assembled into booklets with plastic 
hinges at the top. The new frames in each of the 
Booklets 2 through 8 were preceded by several 
review frames (5 to 17 in number) identical to 
certain frames in the previous booklet. Thus each 
booklet contained from 36 to 45 frames. The 
eight booklets ‘were designed to be used in eight 
class periods over eight successive school days. The 
answer to each item on the front of the card was 
printed on the back. Mimeographed answer sheets, 
with blanks numbered to correspond to frame num- 
bers, were used for the recording of answers by 
the students. 


Performance measures. Designed to be admin- 
istered after the completion of the learning pro- 
gram were three performance measures. The first 
was a performance test of equation solving, con- 
taining 10 simple linear equations of the sort 
encountered in the learning program (see Appendix 
B). This test had a time limit of 25 minutes. 
Scoring allowed for partial correct completion of 
problems, on a scale of 0-4 for each, making the 
maximum possible score 40. The second measure 
was a transfer test (Appendix B), containing 10 
additional linear equations having somewhat un- 
familiar forms and unfamiliar symbols. This was 
a 20-minute test, and was scored in a manner simi- 
lar to that used with the performance test. The 
third test was one designed to measure achievement 
on each of the 22 learning sets identified by the 
analysis previously described. As administered, a 
single item was used to measure each learning set, 
and a total class period of 50 minutes was allowed 
for completion. These items are given in Appen- 
dix A. 


Subjects 


The subjects were members of four different 
mathematics classes, two in the seventh grade of 
Kensington Junior High School, and two in the 
seventh grade of Montgomery Hills Junior High 
School, both located in Montgomery County, 
Maryland. The classes contained students of in- 
termediate abilities, heterogeneously grouped. The 
data of students who missed one or more of the 
sessions involved in the study were eliminated 
from consideration in the results. Of the 144 
students in the four classes, 26 missed at least one 
of the sessions, leaving an N of 118. 


Procedure 


The administration of the tests and learning 
program was carried out by the experimenter in 
each classroom. The classroom teacher remained 
in the room, for the most part, during these 
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sessions. The teacher made assignments of work 
for students to undertake, at their desks, once 
they had completed each booklet of the learning 
program. These arrangements were made before 
each session began The assignments were com- 
posed of unrelated materials. 

During the first class session for each group, 
the tests of basic abilities were administered, 
following the directions given by French (1954) 
The learning program was then administered, 
using the eight booklets, on eight successive class 
days. Three school days intervened between the 
administration of Booklets 6 and 7, owing to the 
interruption of scho | attendance by a severe 
snowstorm. On the day following completion of 
the learning program, the performance test was 
administered in 25 minutes, followed by the trans- 
fer test for 20 minutes On the following day, 
the test of learning sets was given and the total 
period of 50 minutes allowed for its completion 
Actually, there were 44 items on this test, 2 for 
each learning set, but since it was determined that 
these could not be completed, the students were 
instructed to do each odd-numbered item before 
going on to even-numbered ones, and the former 
scores were used in obtaining the measure oO! 
learning set achievement 

In administering the learning program, the ex- 
perimenter first read through the instructions 
which appeared on the first two cards of Booklet 
1. These told the stu lents how to use the booklets 
and how to record their answers. Students were 
instructed to write an answer to the question asked 
in each frame, and then to flip the card over and 
compare their answet with the correct answer 
printed on the back of the card. If their answer 
was right, they were to pr weed; if wrong, they 
were to draw a line through it, turn the card back 
again until they could see how the right answer 
was obtained, and then rec rd the correct answer 
and proce¢ d 

At the beginning of each learning session on 
successive days, the experimenter repeated the 
main points of these instructions The students 
were especially cautioned to record each answer 
before flipping over the card to check the answer. 
(This behavior was monitored throughout by the 
experimenter. ) They were asked to be certain 
that they knew the material on each card before 
proceeding with the next one 


During the administration of each booklet of the 
program, students were instructed to draw a line 
at the edge of their answer sheets under each 
question they had just completed, each time the 
experimenter called “Mark!” This signal was 
given every 3 minutes, although the students were 
not told the amount of this interval. These records 
were used to compute learning rate of each sectior 
of the program A score of learning rate was 
later obtained for each learning set by adding the 
times taken- to complete the frames devoted to 
that learning set, interpolating where necessar) 
Since the program devote d only one frame to eat h 


of the Learning Sets III1 and 114, no learning 
rate score was obtained for these 


RESULTS 


The results of the study may be described 
in accordance with the following organiza- 
tion. First, we shall be interested in report- 
ing the several measures of learning and 
performance that were obtained, which pro 
vide a general picture of the outcomes of 
administration of the learning program. 
Second, a matter of major interest is the 
evidence of transfer among learning sets in 
the hierarchy shown in Figure 1. Do results 
indicate that these learning sets mediate 
positive transfer to higher level learning sets, 
as proposed by the theory? Third, what 
correlations were fe yund among relevant and 
irrelevant basic abilities and total measures 
of performance during and after the com- 
pletion of the learning program? And 
finally, there is particular interest in seeing 
whether patterns of increasing and decreas 
ing correlations are obtained, between abili- 
ties on the one hand and rates of attaining 
‘ndividual learning sets on the other 


Is 


Veasures of Learning and Performance 


The several measures of learning and per 
formance applicable to the entire program 
are shown as means with associated standard 
deviations in Table 1. 


TABLE 1 


MEANS AND STANDARD DEVIATIONS OF SEVERAI 
MEASURES OF LEARNING AND PERFORMANCE 
APPLICABLE TO THE ENTIRE LEARNING 
PROGRAM 

(N = 118 


Measure Vf SD 


Performance on equation solving 

test (10 items, 40 points) 8.4 1.6 
Transfer performance in solving 

unfamiliar equations (10 


items, 40 points 6.1 .-0 
Total number of learning sets 

achieved (22 items) 12.4 3.4 
Time to complete program 

(learning rate) in minutes 221.4 22.4 


It may be seen that in terms of perform- 
ance in equation solving, the learning pro- 
gram was not very successful. This is also 
indicated by the scores on the transfer test 
containing 10 equations of somewhat un- 
familiar form and symbolic content, scored 
with a maximum of 40 points. The mean 
number of learning sets achieved, as meas- 
ured by a test administered following the 
program, was 12.4, another indication of the 
moderately low effectiveness of the program. 


Measures of Transfer among Learning Sets 


It will be recalled that the theory predicts 
high positive transfer from a recalled learn- 
ing set (or sets) at a given level and attain- 
ment of the adjacent higher relevant learning 
set(s). This prediction can be tested by 
noting the pattern of pass and fail which 
obtains between the lower and higher ad- 
jacent sets throughout the hierarchy. It may 
be noted that where two or more subordinate 
learning sets are involved (for example, as 
and are related to I1, Figure 1), 
the prediction is that a// lower relevant sets 
must be passed, in order for the higher level 
set to be attained. 

The four possible empirical relationships 
for passing and failing relevant higher-lower 
learning set combinations are as follows, 
together with the theoretical significance of 
each: 

1. Highert+, Lowert+: This relation indi- 
cates the occurrence of positive transfer from 
lower learning set(s) to adjacent higher 
learning set(s), and is in accord with the 
theory. 

2. Higher—, Lower—: If any relevant ad- 
jacent lower level set has been failed, posi- 
tive transfer to the higher level learning set 
is unlikely; this outcome is also in accord 
with the theory. 

3. Higher+, Lower—: As indicated in 2, 
this outcome is directly opposed to the theo- 
retical prediction. 

4. Higher—, Lowert+: When this outcome 
takes place, it is not in opposition to the 
theory. A higher level set may be failed, 
even though all relevant lower level sets are 
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passed. A number of reasons make this oc- 
currence possible, most of them associated 
with the effectiveness of the learning pro- 
gram. 

The patterns of pass-fail relationships 
indicated by data obtained on the test of 
learning set achievement administered fol- 
lowing the learning program are given in 
Table 2. 

Each of the transfer relationships which 
can be measured between learning sets in 
the hierarchy is listed in the first column of 
this table. As would be expected with a 
program that is not perfectly effective, the 
number of ++ relationships between higher 
and lower learning set combinations goes 
steadily down as the learning program 
carries the learner upward in the hierarchy, 
whereas the number of — relationships in- 
creases. Instances of higher +, lower — are 
cases of positive transfer without the achieve- 
ment of lower level learning sets, and are 
therefore contrary to theoretical prediction. 
It is noteworthy that such instances are very 
low in frequency. The proportion of pass- 
fail patterns which support the theory, ob- 
tained by dividing the total testable instances 
: +—) into the number of instances 
consistent with the hypothesis of positive 
transfer (++, shown in the final 
column. Some of these proportions are 1.00, 
and none is lower than .91. 


(++. -- 


The theoretical prediction is for the values 
in the final column of the table to be 1.00. 
Obviously, they are very nearly that, and 
far above the purely chance values of these 
patterns, which would range between .25 
and .50. One reason, irrelevant to theory, 
for their departure from the 1.00 value is 
of course unreliability of the measurements 
from which they are derived. It will be 
recalled that the measures employed are 
single item pass-fail scores. The use of two 
or more items to assess learning set attain- 
ment would have made possible an estimate 
of the reliability of these measures, but un- 
fortunately these were not used in the 
present study. In future studies, provision 
should surely be made for reliability meas- 
ures. At any rate, on the basis of present 
evidence the prediction of positive transfer 


10 ROBERT M. GAGNE anp NOEL E. PARADISE 


TABLE 2 


Pass-FarL PATTERNS OF ACHIEVEMENT BETWEEN ADJACENT LOWER AND HIGHER LEVEL RELEVANT 
LEARNING SETS, AND THE PROPORTION OF INSTANCES OF POSITIVE TRANSFER INDICATED 
(N = 118) 


Frequency of pass-fail 


Pattern—Higher, Lower Total 
ransfer to testable | Proportion 
learning set | frequency positive 
++ -- +- —+ | (1) + (2) + (3) transfer 
| (1) (2) (3) (4) | 
IV2 from IVAI1 | 110 0 0 8 | 110 1.00 
IV5 from IVA3 } 113 0 0 5 113 1.00 
IIIA1 from IV2, IV3 | 85 0 7 26 92 .92 
IITA2 from IV4, IV5, IV6 94 5 10 u 109 91 
III1 from [V1 45 9 1 63 55 .98 
III2 from IV3, IILA1 68 30 6 14 104 .94 
III3 from IVA2, IV3 75 25 7 11 107 .93 
III4 from IITA2 62 40 4 12 106 } .96 
II1 from IV2, I112, 34 70 3 11 110 .95 
II12 from IVA2, III3 |} 41 60 2 15 103 .98 
II3 from II14 37 72 3 6 112 .97 
II4 from III4 9 85 0 24 94 1.00 
I1 from II1, [12 25 78 2 13 105 .98 
12 from I12, II3 28 80 3 7 111 .97 
I3 from II3, 114 6 104 0 8 110 1.00 


mediated by learning sets appears to be points at which learning was relatively in- 
effective, low values the points at which 
learning was effective. 

The measurement of learning sets makes 
it possible to assess in a fairly exact fashion 
what the individual has learned or has not 
learned from a learning program. Examples 
of the patterns of learning set attainment in 
a low achiever and a high achiever are shown 
in Figure 2 


amply supported. 

The values for the —+ pass-fail pattern 
deserve some comment. As has been said, 
these are not critical evidence for or against 
the theory. What they do indicate, presum- 
ably, is relative weakness in portions of the 
learning program. ror example, 63 of the 
118 learners failed to progress from Learn 
ing Set IV! to Learning Set III1, after 
having achieved the former adequately. Ex 
amination of the program shows that this —7 = 
particular set (III1, identifying needed op- 
erations in order) was covered by only a 
single frame. A similar finding holds for 
Learning Set I14, which has 24 failures after 
"eS success at a lower level. Learning Set IIIA], A 

with 26 failures following lower level suc- 
cesses, is represented by only two frames. ,,, 


The occurrence of these instances suggests 
strongly that the values in the —+ column Fic. 2. Examples of patterns of attainment of 
of Table 2 indicate the points at which _ learning sets in the hierarchy for two individuals, 
a low and a high achiever. (Scores on the final 
oe , performance test of equation solving are indicated 
within the total learning program. High for each individual in the box denoting the final 


values may be interpreted as revealing the task.) 


greater or lesser effectiveness was attained 


5 
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Basic Abilities and Measures of Performance 


Product-moment correlations between the 
basic ability measures and four total per- 
formance measures are given in Table 3. 

On the whole, these coefficients yield an 
expected pattern. Particularly notable is the 
fact that correlations between each of the 
basic variables Number, Symbol Recogni- 
tion, and Integration (measured by tests 
numbered 1, 2, and 3) and each of the four 
performance measures (6, 7, 8, and 9) are 
moderately high. It will be recalled that 
these three abilities are considered to be 
relevant. In contrast, correlations of the 
irrelevant ability measured by Vocabulary 
with these performance variables are fairly 
low, despite the fact that this test is pre- 
sumed to be highly related to general intelli- 
gence. It will also be noted that Letter A 
exhibits higher correlations with perform- 
ance measures than would be expected of an 
irrelevant ability. Although correlations with 
this test are all slightly lower than those for 
the relevant abilities, nevertheless these re- 
sults are not entirely in accord with our 
predictions. Of course, it is possible that 
speed of Symbol Discrimination is, after all, 
a relevant basic ability; but our analysis did 
not predict it to be so. 

Correlations among the performance meas- 
ures themselves are fairly high. In general, 


individuals who did well on the final test of 
performance were those who mastered the 
greatest number of learning sets, and took 
the shortest time to complete the program. 
Achievement on the final test is also highly 
related (.84) to achievement on the transfer 
test; it appears likely that these tests are 
measuring the same capability. 


Relations between Abilities and Learning 
Sets 


Point-biserial coefficients of correlation 
were obtained between each basic ability 
measure (relevant and irrelevant) and pass- 
fail achievement of each learning set as 
measured following learning. Product-mo- 
ment coefficients of correlation were obtained 
between each basic ability measure and rate 
of learning (time to complete the required 
frames, with sign reversed) for each learn- 
ing set in the hierarchy.° For both sets of 
data, interest centered in the pattern of in- 
creasing or decreasing correlation which 
might be revealed as one considers the learn- 
ing sets from the bottom of the hierarchy 
upwards. It will be recalled that the theory 
predicts: decreasing correlation of relevant 
abilities with rates of attaining learning sets 


5 Complete data on intercorrelations of these 


variables are given in Appendix C. 


TABLE 3 


Propuct-MoOMENT COEFFICIENTS OF CORRELATION AMONG TESTS OF Basic ABILITIES ADMINISTERED 
BEFORE LEARNING AND Four MEASURES OF PERFORMANCE 


(N = 118) 


Measure | 1 


. Addition, Subtraction-Multiplication | 918 
. Picture-Number 

. Following Directions | 

. Vocabulary V-1 

. Letter A | 

. Performance test 

Transfer test 

. Number of learning sets achieved 
. Time to complete program (sign reversed) | 


94> 05 22 14 _ 12 18 


51 45.46 50 
84 82° 78 
— 7% 75 
88> 82 

84> 


Note.—Reliabilities shown in diagonal. 
Correlation between Addition and Subtraction- Multiplication. 
» Odd-even split-half correlation. 
Unavailable. 


: 

: ; 1 41 33 04 38 68 6 £58 ~ 55 

; 
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f as one progresses upwards; increasing cor- sets begin with IVA1, 2, and 3; at Level IV, 
relation of relevant abilities with achieve- however, only Learning Sets V2, 1V4, 1V5, 

ment of learning sets, proceeding in the and IV6 are relevant (IV1 and IV3 are 
: same direction; and moderately low and _ connected to Symbol Recognition neither by 
constant correlations of irrelevant abilities an arrow nor by an intermediate learning 

(and general intelligence) with both learning — set at Level IVA). At higher levels, all the 

rate and achievement. learning sets are relevant to Symbol Recog 

Examination of Figure 1 will reveal how _ nition except for III1, since they are con 
the relevant learning sets were selected for nected to this ability via intermediate learning 


this treatment of the data. For Symbol _ sets. For Number Ability, relevant learning 


4 Recognition ability, the relevant learning sets begin with IV3 and IV5; again, all 
B 
z 
a 
= 
z 
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oO 
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uJ 
N 
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-—LEVEL— 


Bk 3a. The relationships with learning rate, based 3b. The relationships with achievement, based on 
point biserial coethcients 


IVA WIA 


upon product-moment correlations 


Fic. 3. Median values of coefficients of correlation of relevant basic abilities (Number, Symbol 
Recognition) and irrelevant basic abilities (Speed of Symbol Discrimination, Vocabulary) with meas 
ures of learning sets at different levels of the learning set hierarchy. (For irrelevant sets. medians of 
the highest three coefficients are shown, except for Level IITA which contains only two, In all cases, 


straight lines have been fitted to the sets of points by tye.) 


a 
a 
| 
| > 


others are relevant except for III1. For 
Integration, the only relevant sets are IV1, 
I111, and 12, and these data were not plotted 
because of the sparsity of points which could 
be determined. 

igure 3 graphs the relationship of the 
size of the median correlation obtaining be- 
tween relevant basic abilities and learning 
sets and between irrelevant basic abilities 
and learning sets, to the level of the hier- 
archy in which the sets are located. Figure 
3a pertains to learning rate; Figure 3b, to 
achievement of learning sets. In order to 
provide an idea of the slope of the relation- 
ships, straight lines have been fitted to the 
points by eye. The median correlation at 
each level was determined from values of 
the particular number of relevant learning 
sets at that level. lor the irrelevant sets, 
the median of the three highest correlations 
was taken (except for Level IIIA, where 
only two exist), in order to have the number 
of values on which to base a median com- 
parable to those for relevant sets. 

Considering the data shown in [igure 3a, 
it is apparent that there is quite a marked 
difference between the changes that occur in 
the correlations between relevant sets and 
learning rate, and those between irrelevant 
sets and learning rate, as one progresses up- 
wards in the hierarchy. Relevant abilities 
are correlated highly at low levels of the 
hierarchy, and markedly decrease in correla- 
tion at high levels. In contrast, the changes 
in correlation with irrelevant abilities, while 
not absent, exhibit an extremely small slope. 
These results, then, tend to confirm the 
theoretical prediction that rate of learning 
depends decreasingly upon relevant abilities 
as learning progresses upwards in the hier- 
archy. In addition, irrelevant abilities exhibit 
moderately low and nearly constant values 
of correlation with the rate of attaining 
learning sets. 

The picture for correlations of abilities 
with achievement of learning sets (T*igure 
3b) is nearly the opposite in every respect. 
Here, there is a pattern of increasing cor- 
relation between relevant abilities and learn- 
ing set achievement. The explanation of 
this trend is not crucially dependent on 
theory in this case, but is based on the rea- 
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soning that increasing numbers of individuals 
effectively “drop out” as learning proceeds; 
thus the variance which remains becomes 
more clearly that of relevant basic abilities. 
Again, in the case of irrelevant abilities, the 
change in the size of correlation is not nearly 
so great. The smallest slope is obtained for 
Vocabulary, often considered to be a meas- 
ure of general intelligence, while Speed of 
Symbol Discrimination attains an intermedi- 
ate slope. On the whole, the contrast between 
the behavior of relevant and irrelevant sets 
is a marked one, nearly as striking as that 
which obtains for learning rate (Figure 3a), 
but in the opposite direction. 

The correlations obtained between learning 
rate and Integration were .76 with IV1 and 
43 with 12; it will be recalled that the 
learning rate of III1 could not be measured. 
This test, too, shows the decreasing pattern 
found with other relevant abilities. For 
achievement of learning sets, the values of 
the correlations were .53, .50, and .56 at 
Levels IV, III, and I, respectively. These 
data do now show the kind of increasing 
pattern which obtains for the other relevant 


abilities. 


Relations among Learning Sets 


If the rate of learning for progressively 
higher learning sets in the hierarchy depends 
decreasingly upon relevant basic abilities, 
then, according to the theory, it must come 
to depend increasingly upon attained knovel- 
edge, that is, on the successful achievement 
of relevant subordinate learning sets. <Ac- 
cordingly, the data were next examined for 
evidence of this latter relationship. 

Table 4 contains median point-biserial co- 
efficients of correlation for relationships 
between achievement of each learning set 
and rate of learning of relevant adjacent 
higher learning set or sets. These are com- 
pared in each case with correlations between 
achievement of the same learning set and 
irrelevant adjacent higher learning sets. ° 
The table indicates the adjacent higher sets 
from which the median 


in each instance 


6 The individual correlation coefficients are given 


in Appendix C. 
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4 


14 ROBERT M. GAGNE anv NOEL E. PARADISE 


correlation was derived; the distinction be- 
tween relevant and irrelevant sets was of 
course derived from the theoretically pre- 
dicted connections shown in Figure 1. (This 
analysis of the data begins at Level IV, 
since achievement of learning sets at Level 
IVA was virtually perfect, and thus provided 
no variance. ) 

Generally speaking, the correlations of 
achievement and learning rate for relevant 
pairings show a moderate rise as one goes 
upwards in the hierarchy. In addition, each 
of the correlations for relevant pairs is 
higher than the corresponding correlation 
for irrelevant pairs. Tests of significance of 
these differences (utilizing the median in- 
tercorrelations shown in the next-to-last 
column to obtain a value of ¢ applicable to 
correlated samples) are shown in the final 
column of the table. Half of these com- 
parisons attain a satisfactory level of signifi- 
cance, while half do not. At the higher 
levels of the hierarchy, all the comparisons 
are significant ones, indicating that learning 
rate at these levels is dependent upon achieve- 
ment of adjacent subordinate learning sets. 


This finding is to be contrasted particularly 
with the. previous one concerning the rela- 
tions between basic abilities and learning 
rates; it will be recalled that for these 
variables, relationships pertaining to relevant 
ones become indistinguishable from irrele- 
vant ones at higher levels of the hierarchy. 
It does appear, therefore, that rate of attain- 
ment of learning sets comes to depend in- 
creasingly upon knowledge achievement (as 
indicated by successful achievement of ad- 
jacent subordinate learning sets), and de- 
creasingly upon amounts of basic abilities. 


DIscuSSION AND IMPLICATIONS 


The results have indicated, first of all, 
that the acquisition of individual capability 
in solving linear equations, established by 
a learning program, may be conceived as a 
matter of attaining a hierarchy of learning 
sets which may collectively be called knowl- 
edge. A final task of this sort may be 
analyzed to reveal a supporting hierarchy of 
learning sets by asking the question: “What 
would an individual have to know how to do 


TABLE 4 


MEDIAN COEFFICIENTS OF CORRELATION (fppi) OF RELEVANT AND IRRELEVANT PAIRINGS COMPRISING 
LEARNING SET ACHIEVEMENT VS. LEARNING RATE OF ADJACENT HIGHER LEARNING SET(S), WITH 
SIGNIFICANCE OF DIFFERENCES BETWEEN THESE COEFFICIENTS 
(N = 118) 


Irrelevant pairing 


Achievement 


— Mdn. of 


on learning 


Tpbi difference 


set: Learning rate of: Mdn.| Learning rate of: Mdn. 


(1) (2) 


(3 ) Tpbi 


(2) vs. (3) (r12-7 13) 


1V2 Ili; 


112, 3; IITA2 


IV3 1112, 3; | 
1V4 34 | 
IIIA2 | 
IILA2 35 | 
IIIA1 1112 1113, 4 
1114 1112, 3 
1112 Ili 38 | I12,3 
1113 Il, 2 113 
1114 113 Il1, 2 
Ili Ii : 12, 3 
112 Ii, 2 13 

113 12, 3 Ii 

114 13 | 11,2 


Relevant pairing Po 
Tpbi 
| | | 
43 25 30 >.05<.10 
’ 19 25 <.05 
18 26 >.10 
ae 19 26 >.10 
21 26 >.10 ; 
21 29 >.10 
= 22 28 >.10 
22 40 > .05<.10 
18 29 <.02 : 
2 19 29 <.05 
30 | 37 <.01 
25 | 36 <.01 
24 37 <.01 
30 36 <.01 


in order to perform this task, after being 
given only instructions?” The answer to the 
question defines immediately subordinate 
learning sets, and the question may then be 
applied to these tasks in turn to define the 
next subordinate level of learning sets. By 
theory, each learning set is conceived to 
function by mediating positive transfer to a 
higher level task or tasks. 

In this study, such an analysis defined 22 
subordinate learning sets arranged in hier- 
archical fashion. When the question de- 
scribed previously was applied to those sets 
occurring at the next to the lowest level, the 
answer defined the additional tasks of Num- 
ber, Symbol Recognition, and Integration, 
which appeared to be the same as those used 
in certain factor-reference tests. 

Relationships among the variables meas- 
ured in the study lead to the conclusions that: 
(a) a high incidence of positive transfer 
(nearly 100% of the instances tested) ob- 
tains from success on relevant subordinate 
learning sets to attainment of a superordinate 
learning set; (>) a decreasing pattern of 
correlations can be shown between relevant 
basic ability factors and rate of learning 
for learning sets as one progresses upwards 
in the hierarchy; (c) in contrast, correla- 
tions of rate of learning’ with irrelevant 
ability factors show only a slightly decreas- 
ing trend; (d) learning rate of the learning 
sets in the hierarchy depends increasingly 
on acquired knowledge (in the form of 
subordinate learning sets) and decreasingly 
upon basic ability factors as one proceeds 
from the bottom to the top of the hierarchy. 


Implications for Ability Measurement 


These findings may have important impli- 
cations for the measurement of abilities 
involved in any human task. For one thing, 
they emphasize the importance of measures 
_ of rate of learning as criteria against which 
the predictive efficiency of an aptitude test 
may be assessed. There appears to be a 
fairly clear rationale for relationships be- 
tween an initial ability score and rate of 
learning of a relevant task, if one assumes 
that the ability represents a level of capa- 
bility which mediates a greater or lesser 
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amount of positive transfer. Basic abilities, 
or factors, may accordingly be conceived as 
the most simple and most general learning 
sets, which support a great variety of more 
complex activities. Such support manifests 
itself as greater or lesser facilitation of the 
rate of learning or related tasks. 

The suggestion is, therefore, that a true - 
integration of measures of productive learn- 
ing and aptitude measures can be effected 
by identifying and correlating rate of learn- 
ing of learning sets in a hierarchy with basic 
ability factors that are defined by means of 
the same analysis. Such a procedure would 
make possible the testing of hypotheses in 
aptitude studies, in much the same fashion 
as has been done in the present study. It 
would also reveal a rational basis for the 
statement that human tasks “depend upon” 
or “are related to” aptitude factors. And 
particularly, such studies should make pos- 
sible a suitable determination of the relative 
contribution of knowledge variables, as com- 
pared with basic ability variables, in the 
achievement of performance on a final task 
(representing a class of tasks). 

The correlation of basic ability factors 
with achievement, our results suggest, is 
really a more complex matter for which the 
rationale is less clear. Specifically, the 
achievement of any given task (which may 
be any learning set in a hierarchy) depends 
not only upon the amount of basic ability, 
but also upon the amount and kind of spe- 
cifically transferable knowledge that has 
been acquired. As learning proceeds to 
higher level learning sets, relevant abilities 
will correlate to an increasing extent with 
achievement only to the extent that the 
learning program is ineffective in mediating 
transfer from one level of the learning set 
hierarchy to the next. A “power” test of 
achievement, containing items of increasing 
complexity, may be conceived as a deliber- 
ately inefficient learning program, whose 
correlation with basic ability factors depends 
upon the fact that increasing numbers of 
people “drop out” as they attempt more and 
more items of the test. This is, of course 
a perfectly good rationale for such a test. 
But it does fail to distinguish between the 
contributions made to performance by knowl- 
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edge factors as opposed to ability factors. 
For precise prediction of performance on 
an achievement test, one must measure the 
recallability of subordinate learning sets, 
rather than search for additional basic 
abilities. 


Learning Programing 


As has been pointed out in a previous 
article (Gagné, in press) the theory of 
learning set hierarchy has a number of im- 
plications for the programing of productive 
learning. Chief among these is the idea of 
designing the frames of a program in such 

way that they: constitute an ordered se- 
quence logically related to the hierarchy of 
learning sets for the desired final task, pro- 
vide for recallability of subordinate learning 
sets, and furnish the guidance to thinking 
which will enable the learner to integrate 
subordinate learning sets in the performance 
of new tasks. The learning program used 
in the present study was not constructed 
according to these principles. Nevertheless, 
the evidence has some things to say about 
them. 

Although the program was not deliber- 
ately designed to establish the learning sets 
identified by theoretical analysis, the evi 
dence indicates that this 1s what it did do, 
to the extent that it was effective. Further 
more, there is good evidence that when the 
particular learning sets required for new 
learning were present in the individual, high 
positive transfer resulted; when they were 
absent, very low transfer took place (cf 
Table 1). It is also significant to note that 
the analysis, carried out according to theory, 
was able to identify three learning sets which 
were inadequately emphasized within the 
program (Learning Sets IITA1, ITI1, and 
[14, Figure 1) in the sense that only one or 
two frames was devoted to each. The evi 
dence suggests that transfer was particularly 
low at just these points in the program. It 
seems reasonable to conclude that the anal) 
sis of a final task into subordinate learning 
sets can be an important, if not essential, 
first step in the development of an effectiv: 


learning program 


The use of learning set analysis to identify 
the required sequence of a learning program, 
and later to measure its effectiveness, can 
provide a solution to a measurement diffi- 
culty encountered by methods in common use 
at present. On logical grounds, internal cri- 
teria of program effectiveness such as time 
to complete frames or number of errors per 
frame, cannot be considered convincing evi- 
dences of how well a program is accomplish- 
ing what it is designed to do. Measures of 
final performance or of transfer are beyond 
doubt best suited for this purpose. However, 
the latter measures are not particularly 
diagnostic of the relative effectiveness of 
portions of the learning program. The 
measurement of learning sets has the advan 
tage of accomplishing both these purposes: 
it can diagnose weak points, and it can 
measure the achievement of the learner up 
to any point at which he “loses under- 
standing,” or effectively “drops out.” The 
evidence of this study shows that the meas 
urement of learning sets can provide the 
experimenter with relatively precise infor 
mation about the progress of the individual 
throughout the course of learning 


SUMMARY 


An analysis was made of the class of tasks 
“solving linear algebraic equations,” follow 
ing the outline of a learning program de 
signed to establish proficiency in these tasks, 
to identify a hierarchy of learning sets which 
support the attainment of such proficiency. 
This analysis was based upon a theory to 
the effect that attainment of any given 
learning set is dependent on recallability of 
certain subordinate learning sets, instructions 
defining the stimuli and goal of the new task, 
and integration by the learner of subordinate 
learning sets into the solution of the new 
task. Subordinate learning sets are conceived 
as having the function of mediating positive 
transfer to higher level learning sets through 
out the hierarchy, and ultimately to the final 
task. Subordinate learning sets for a given 
class of tasks may be defined as the answer 
to the question: “What would the individual 
have to be able to know how to do, in orde1 
to be able to perform this (new) task, being 
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final task, the question is applied successively 
to each learning set so defined, and thus 
identifies a progression of learning sets which 
grow increasingly simple and increasingly 
general. 

Besides defining a hierarchy of 22 learn- 
ing sets, three additional ones, represented 
by very simple tasks, were derived from this 
analysis at the lowest level of the hierarchy. 
These appeared to be identical to three tasks 
occurring in ability tests which have been 
identified as relatively “stable” in factorial 
studies, the so-called “factor-reference’”’ tests. 
Specifically, these tasks appeared to be those 
involved in tests measuring Number Ability, 
Symbol Recognition (a particular form of 
Associative Memory), and Integration I (an 
ability apparently involving keeping several 
procedures in mind at once, as in “following 
directions”). The occurrence of these abili- 
ties as an end result of a theoretical analysis 
of the knowledge composition of a learning 
program naturally raised the interesting 
question as to how they function in support 
of the learning of the final performance, as 
well as of the intervening learning sets in 
the hierarchy. 

According to theory, basic abilities which 
are relevant to learning sets in the hierarchy 
should mediate positive transfer to them, 
and this in turn should be measurable as an 
increased rate of learning. In progressing 
upwards in the learning set hierarchy, cor- 
relations of these abilities with rate of 
attainment of relevant learning sets should 
decrease, since such relations come to depend 
increasingly upon transfer from immediately 
subordinate learning sets (i.¢., upon specific 
knowledge). General intelligence, while it 
may be expected to correlate to a moderate 
degree with rate of learning throughout the 
hierarchy, should show no change with po- 
sition of learning sets at various levels of the 
hierarchy, since it is conceived to mediate 
general, rather than specific, transfer. The 
same is true for basic abilities which are 
irrelevant to the learning sets to be learned, 
in the sense that they have not been con- 
nected to them by analysis. As for correla- 
tions of basic abilities with achievement 
(pass-fail) of relevant learning sets, the 


given only instructions?’ Beginning with the 
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prediction would be that these would exhibit 
an increasing pattern as one progresses up- 
wards in the hierarchy. The reasoning is 
that, to the extent that the learning program 
is ineffective in reducing all individual dif- 
ferences in achievement, increasing numbers 
of individuals “fail to grasp” the program 
as it proceeds; consequently, the relationship 
comes to depend increasingly on a differ- 
entiation of high from low ability. Thus 
increasing correlations of achievement with 
relevant basic abilities provide an inverse 
indication of the effectiveness of the learning 
program. If the program were perfectly 
effective, differences in achievement would 
of course not be measurable. 

To test these predictions, a study was 
conducted by administering the learning pro- 
gram on equation solving to a group of 118 
seventh graders in four different school 
classes. The program was divided into eight 
booklets, and administered during eight suc- 
cessive class days in the school room. Pre- 
ceding this administration, factor-reference 
tests were given to obtain measures of basic 
abilities, both relevant (Number Ability, 
Symbol Recognition, Integration I), and ir- 
relevant (Speed of Symbol Discrimination, 
Vocabulary). During the learning program 
administration, students were required to 
mark the margins of their answer sheets at 
3-minute intervals, to provide the basis for a 
measure of rate of learning. Following com- 
pletion of the program, in another class pe- 
riod, two 10-item tests were administered to 
measure performance in equation solving, 
and transfer to the solving of equations hav- 
ing somewhat unfamiliar symbols and form. 
I‘inally, a test was given to measure achieve- 
ment of the 22 learning sets in the hierarchy. 

Predictions were confirmed in the follow- 
ing respects: 

1. Correlations of theoretically relevant 
basic abilities were higher than those of 
irrelevant basic abilities with measures of 
final performance, with transfer of training 
scores, with number of learning sets achieved, 
and with rate of learning of the total pro- 
gram. 

2. Instances of positive transfer to each 
learning set from subordinate relevant learn- 
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ing sets were found to occur throughout the 
hierarchy with proportions ranging from .91 
to 1.00. 

3. Correlations of relevant basic abilities 
with rates of attainment of learning sets at 
progressively higher levels of the hierarchy 
showed a steeply progressive decrease. In 
contrast, the pattern of correlations of irrele- 
vant basic abilities with the rates of attain- 
ment of comparable learning sets remained 
nearly constant. 

4. Correlations of relevant basic abilities 
with achievement of learning sets at progres- 
sively higher levels showed an increasing 
pattern, whereas the comparable pattern with 
irrelevant abilities exhibited at the most a 
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APPENDIX A 


Tasks Usep IN THE MEASUREMENT OF LEARNING SETS 


IVA1: Equivalence of 1x and x 
Which term is the same as x? 3x; 10x; —2x; y; 1x; 8x; —y 


IVA2: Identification of an Equation 
Which of the following is an equation? 


8y + 2 — 10 (8y + 2)10 
8y +2 
8y +2 = 10 10 


IVA3: Obtaining Products with Zero 
Which of the following are equal to zero? 


52-3 4r-0 
2-11 16-8k 
0-157 7f-7e 


IV1: Procedural Order 
The order to be followed in solving an equation is: 


LCRD C—Collect like terms 

RCLD D—Divide by the coefficient of ‘‘x"’ 
LRCD R—Transpose all other terms to the right 
DRCL L—Transpose all ‘‘x’’ terms to the left 


IV2: Recognition of Equivalent Terms 
Which term on the right can be combined with (added to or subtracted from) each term on the left? 
For every term on the left there is one term on the right which can be combined with it. 


1. 3(a — dD) 14s 9q 5m —6n 
2. 12n Sp 40 —4a lpr 
4. : 16a (x — y) 
5. 16 


IV3: Performing Addition and Subtraction of Numbers in Sequence 
34+9+27 —-134+2 68 = 


IV4: Recognizing Equivalence of Multiplication and Division Terms 
Which three terms mean 34a multiplied by bc? 


34a + be 
34ab-c 
34a = be 


1V5: Performing Multiplication of Numbers in Sequence 
Multiply: 2, 8, 6 


1V6: Division of Parenthetical Terms 
7(2a + b — 5) 
3b(2a + 6 — 5S) 


II1Ai: Combining Fractions with Like Denominators 
3 ee 2 = 
Simplifying Fractional Expressions 
3-8-4-9-60 
10-3-12-2 
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34a-be = 
bc 
34a — be 34abc 
a 
— 4a + be 
34 34a 
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III1: Identifying Needed Operations in Order 
Given this equation and the steps in its solution (in mixed-up order), place the steps in their proper 
order and label the operation to be done on each step (in order to get to the next step). Use the steps listed 
in Question 4. 
Equation: 4x — 2 =x +3 Steps: 3x =5 
4x — <x - 
4x 


1112: Addition and Subtraction of Terms in Sequence 


2y + 7s — 8y — 4w + + — = 


I113: Supplying Sum and Difference Equivalents to Sums and Differences (Arithmetic Numbers 


247-S+1-6=3-6+8+2+? 


1114: Supplying Product and Quotient Equivalents to Products and Quotients (Arithmetic Numbers 
2-4-3 2-3-4 


= ? 


6-2 3-6 


: Supplying Sum and Difference Equivalents to Sums and Differences (Terms 


3g + Sf +h = 2g — 3f —4h + 


2: Simplifying an Equation by Adding and Subtracting Arithmetic Numbers to Both Sides 
Solve for w: 146+w—-10 =7+4-2 

3: Simplifying an Equation by Multiplying and Dividing Both Sides by Arithmetic Numbers 
Si 


Solve for ¢ 7 = 10 


: Supplying Product and Quotient Equivalents to Products and Quotients (Terms 
3s°5r-q s-4q 
10 10r 


: Simplifying an Equation by Adding and Subtracting Terms to Both Sides 
Solve for 6 76 + 2a + 3b —a 10b + 2a + 35 b 


: Simplifying an Equation by Multiplying, Dividing, Adding, and Subtracting Arithmetic Numbers 


Solve for a: 20 = 


3: Simplifying an Equation by Multiplying and Dividing Both Sides by Terms 
6(3 — e) 


> 7/2 
— €)€ 


Solve for e: 
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APPENDIX B 


Test for Equation Solving 
1. Solve for b: 2b — 3 — 8b — 4 + 3b = 13 — 6 — 3b — 2 — 6b 


2. Solve for z: ice ears 


8 
3. Solve for a: * 52 —12 = ar — 2a 
4. Solve for y: = 12 
5. Solve for m: om -=m-—2 
6. Solve for x: = = 6x — 8 
7. Solve for a: * + “ +b=2a+4 
8. Solve for p: =14-4+2 


9. Solve for x: 7x + 4x = 3a+3a+2a-—x 
10. Solve for g: 2g —-3h +2+g =7 — 8h — 2g 


Transfer Test (Equations) 


(F + 
oe 1. Solve for F: “ +4) ,GF—8) 
+ (4 +3L) +6 + (41-0) _ 3L-4-2-14 


2. Solve for L: 


3 

3. Solve for p: —2+3p+3¢ —2r = 18 +49 -64+3p-—,r 

4. Solve for #: i- 

5. Solve for g: 4g + 12 
3N+N—2 
iV: = i( — 12 
6. Solve for N N+2N—4 10 + 4 1 
6a — 6b 


7. Solve for c: bt 5 —2 


8. Solve for T: 4Tx — 4Ty = (Sx + 3y)(x — y) 


9. Solve for Q: + 30 
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APPENDIX C 


TABLE C1 


COEFFICIENTS OF CORRELATION BETWEEN BASIC ABILITIES AND RATES OF ATTAINING LEARNING SETS 
(PRopUCT-MOMENT) AND BETWEEN THE SAME ABILITIES AND ACHIEVEMENT OF LEARNING SETS 
(Point-BIsERIAL) 


(N = 118) 


Ability factors 


Learning rate Achievement 


Learning 
set Speed Speed 


: Symbol _Inte- of Sym- Symbol Inte- of Sym- 
Recog- gration bol Dis- Recog- gration bol Dis- 
ber ulary er ulary 
nition I crimi- nition I crimi- 
nation nation 


76 30 28 40 20 53 02 28 
57 26 18 30 33 30 02 47 
49 28 28 | 24 29 05 01 11 
45 26 a ie 22 11 08 34 
7 40 32 40 31 25 14 09 21 
IV6 69 69 38 24 28 | 26 23 20 06 0 
IIIA! 64 58 40 22 40 26 52 34 22 33 
IITA2 74 55 34 20 33 49 50 26 14 29 
lil 50 49 50 08 42 
1112 44 54 34 18 30 59 51 41 24 37 
III3 60 55 25 21 24 41 35 27 09 $3 
IIl4 62 34 28 24 26 68 60 43 15 47 
52 33 7 11 
42 44 25 49 

59 50 5 57 
63 34 09 23 
57 41 11 36 
71 56 0 53 

30 
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: IVAI | 38 80 60 26 40 
IVA2 | 40 75 54 24 28 | 
IVA3 | 71 76 51 28 33 
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TABLE C2 


COEFFICIENTS OF CORRELATION (POINT-BISERIAL) BETWEEN ACHIEVEMENT OF EACH LEARNING SET AND 
LEARNING RATE OF EACH HIGHER-LEVEL LEARNING SET | 


(N = 118) 


Achievement Rate of learning 


of learning 
set HIA2 ~~ III3 III4 Ili 


20 28 31 34 25 26 
38 18 44 24 32 48 
39 18 * 43 46 19 49 
18 34 22 20 44 19 
19 33 21 19 45 18 
21 35 19 17 46 16 
32 24 18 42 

24 20 35 21 

22 

38 

38 

18 
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Iv2 30 25 
IV3 50 24 S58 44 26 
IVS 24 «49 
IV6 26 48 «#425 52 66 
4 22 18 25 
IITA2 18 41 26 52 37 
112 2 18 SS) 24 26 
1113 46 18 SS 44 26 
It 55 
112 53 
114 2 
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